Audio documents

Most documents hold textual knowledge that is split into chunks and retrieved to ground the agent's responses. An audio document is different: it stores one or more pre-recorded audio files for playback during a call, and is not split into chunks or used for retrieval.

Audio documents are referenced by name and used to:

• play fixed prompts from a Flow's Say conversation node, via the #play_file and #play_digits directives – see Playing pre-recorded audio.
• replace specific LLM-generated phrases of an Agent with a recording, via the prerecorded_audio advanced configuration parameter – see Play prerecorded audio files instead of specific LLM responses.

Playing a recording instead of synthesizing speech reduces TTS usage and cost, and avoids mispronunciation of known, fixed phrases.

Preparing the audio files

• Use the wav/lpcm16 format where possible. Other formats supported by the Voice.AI Connect playUrl API are also accepted – see the play_url pre-defined tool for the full list.
• Each file's extension must be .wav or .pcm.

• For example, to convert a file to the recommended format with ffmpeg:

  ffmpeg -i input_file.ext -acodec pcm_s16le output.wav

Uploading the files as a document

1. In the Navigation pane, expand AI Agents, and then click Documents.
2. Click Add new document.
3. In the Name field, enter a unique name (up to 32 characters; letters, digits, or underscores only). This is the name you reference when playing the audio.
4. Select Upload File, then click Upload file. In the file picker, change the file-type selector to All files so that .wav / .pcm files become selectable, and choose the audio files you prepared (and, optionally, their transcript files).
5. Click Create.

How files within a document are selected

• If a document contains multiple audio files, a random file is played each time the document is referenced without a specific file name. This is useful when you have several variants of the same phrase – for example, different recordings of a welcome message.
• To address a specific recording, reference it by file name – for example #play_file <document> welcome.wav. The .wav / .pcm extension may be omitted; the matching file is resolved automatically.
• For #play_digits, the document must contain one file per digit, named 0.wav … 9.wav (or .pcm).
• If you need different recordings for different phrases, create a separate document for each.

Transcript files

Alongside the recordings, you can upload a transcript of each audio file – the text that the recording speaks. When the file is played – from a Flow Say node's audio playback or an agent's prerecorded_audio playback – its transcript appears in the conversation log alongside the played audio, so you can see what each recording said.

• A transcript file must share the same base name as its audio file, with a .txt or .json extension – for example welcome.txt (or welcome.json) for welcome.wav, or 0.txt for 0.wav.
• Each audio file may have at most one transcript – either .txt or .json, not both.
• A transcript must correspond to an audio file in the same document; a transcript with no matching recording is rejected.
• Removing an audio file also removes its matching transcript file.
• Transcript files are not played and are not split into chunks; they only supply the log text for the played recording.

Two formats are supported:

• .txt – the file content is the transcript text.

• .json – a list of word entries, each with a word field; the word values are joined with spaces to form the transcript. This matches the word-level output of the AC DNN speech-to-text engine, for example:

  [
      { "word": "your", "confidence": 0.999, "duration": 39, "location": 9 },
      { "word": "reference", "confidence": 0.998, "duration": 51, "location": 48 },
      { "word": "number", "confidence": 0.997, "duration": 46, "location": 99 }
  ]

Upload the transcript files together with the recordings (in the file picker, use the All files type so that .txt / .json files are selectable). The info pane's View action shows a transcript file's actual content.